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SUMMARY  PAGE 


THE  PROBLEM 

To  provide  an  interim  report  summarizing  the  first  fifteen  months 
of  Sea  trials  experience  with  the  Computer-Assisted  Medical  Diagnosis  of 
Abdominal  Pain  Project;  Work  Unit  //63706N-M0095 . 001-1045 . 


THE  FINDINGS 

Results  of  the  first  fifteen  months  of  sea  trials  experience  show 
good  correlations  between  the  computer  diagnosis  and  both  the  submarine 
Hospital  Corpsman's  initial  diagnosis  and  the  final  diagnosis  in  the 
limited  number  of  cases  reported.  The  datasheet  has  been  particularly 
useful.  However,  unless  the  percentage  of  presenting  abdominal  pain  cases 
that  are  reported  increases  substantially,  it  is  unlikely  that  the  computer 
model  will  be  validated  at  the  end  of  the  five-year  trial. 


APPLICATIONS 

The  information  will  be  useful  to  those  following  the  progress  of 
this  research  in  computer-assisted  medical  diagnosis  in  isolated  environments. 


ADMINISTRATIVE  INFORMATION 

This  report  was  submitted  for  review  in  November  1983,  and  was 
approved  for  publication  in  December  1983.  It  is  designated  as 
NAVSUBMEDRSCHLAB  Report  Number  1013. 


Published  by  the  Naval  Submarine  Medical  Research  Laboratory 
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ABSTRACT 


This  report  summarizes  the  experience  with  the  Computer-Assisted 
Medical  Diagnosis  of  Abdominal  Pain  project  from' 1  July  1982  -  30  September 
1983-  During  this  period,  30  cases  of  abdominal  pain  presenting  at  sea  were 
reported  to  the  Naval  Submarine  Medical  Research  Laboratory.  Seventy-three 
percent  of  these  came  from  the  Experimental  Group,  twenty-seven  percent  from 
the  Control  Group.  Review  of  sick  call  logs  and  personal  discussions  with 
submarine  Hospital  Corpsmen  disclosed  that  80-90%  of  cases  went  unreported, 
largely  because  of  the  paperwork  involved  and  concern  for  uses  to  be  made  of 
the  data.  Personal  visits  to  submarine  Hospital  Corpsmen  were  made  to  allay 
these  concerns. 

The  datasheet  was  found  helpful  by  all  who  used  it.  The  computer 
program  supported  the  Medical  Department  Representative's  initial  diagnosis 
75%  of  the  time  or  better,  and  was  appreciated  when  used. 

The  potential  value  of  this  project  as  a  sophisticated  decision  aid 
to  the  submarine  medical  department  representative  is  underscored  by  the 
results  to  date.  A  rigorous  scientific  trial  to  validate  the  computer 
program  is  necessary  for  medical  safety.  At  present,  the  outlook  for 
eventual  validation  is  gray  because  of  insufficient  case  reporting. 
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COMPUTER-ASSISTED  DIAGNOSIS  PROGRAM  FOR 
ACUTE  ABDOMINAL  PAIN 


I.  BACKGROUND 

The  Naval  Submarine  Medical 
Research  Laboratory  (NSMRL)  is 
tasked  with  developing  new  methods 
and  procedures  for  assisting  the 
medical  department  representative 

(MDR)  aboard  deployed  United 
States  Navy  submarines.  Paramount 
among  the  duties  of  the  MDR  is 
the  evaluation  and  treatment  of 
crewmembers  who  are  ill .  The 
history  and  physical  exam  are  the 
cornerstone  of  the  evaluation, 
with  laboratory  facilities  being 
modest.  Consultation  with  other 
medical  personnel  is  attempted 
only  in  emergencies  and  if  the 
mission  permits  it.  With  the 
knowledge  that  computer-assisted 
diagnosis  was  under  study  else¬ 
where,  NSMRL  saw  an  opportunity 
to  develop  a  computerized  diagnos¬ 
tic  aid  for  use  by  the  MDR. 

Since  abdominal  pain  was  the 
most  frequent  chief  complaint  in 
medical  evacuations  from  sub¬ 
marines,  a  computer  program  to 
assist  in  developing  a  differential 
diagnosis  of  this  condition  was 
initiated  several  years  ago. 

The  intent  was  (and  is)  to  provide 
a  "sophisticated  decision  aid"  to 
supplement  the  modest  laboratory 
facilities  and  to  compensate  for 
the  lack  of  a  consultant. 

A  prototype  program  was 
formulated  in  conjunction  with 
Dr.  F.  T.  de  Dombal  from  the 
University  of  Leeds  Hospital  in 
the  United  Kingdom.  It  was 
modified  for  the  submarine 
population  and  made  ready  for  sea 
trials.  Proper  scientific  and 
medical  protocol  required  that  the 


project  be  conducted  as  an 
experiment  and  that  a  control  (C) 
and  experimental  (E)  group  be 
formed.  A  protocol  was  approved 
by  the  Committee  for  the  Protection 
of  Human  Subjects  which  involved 
informed  consent  for  participation 
from  MDR's  and  their  randomization 
into  a  control  and  experimental 
group.  The  Control  group  has  the 
datasheet,  reference  manual,  and 
(recently)  the  training  program. 

A  five-year  study  period  was 
projected  from  the  anticipated 
case  load  so  that  statistical 
requirements  could  be  met.  The 
case  load  was  estimated  from 
incidence  data  for  abdominal  pain 
supplied  by  MDR's.  General 
descriptions  of  the  program  have 
been  previously  reported . 1-4 

In  January  1982  approval  was 
granted  for  Fleet  T&E  support. 

In  May  1982  training  of  Squadron 
Medical  Officers  and  Squadron 
Corpsmen  was  accomplished  with 
concomitant  distribution  of 
computer  tapes  and  instructional 
materials.  Individual  submarine 
corpsmen  were  then  trained  by 
Squadron  Medical  Personnel.  In 
July  1982  sea  trials  were  initiated. 
This  report  summarizes  results  from 
July  1982  through  September  1983. 

II.  DATA  COLLECTED 

One  hundred  and  three  submarines 
are  actively  enrolled  in  the 
study,  with  52  in  the  control 
group  and  51  in  the  experimental 
group.  The  enrollment  numbers 
fluctuate  with  additions  to  the 
fleet  and  decommissionings  (or 
transfers  to  the  shipyard).  To 
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date,  there  have  been  30  cases  of 
abdominal  pain  reported  to  NSMRL. 
Twenty-two  (73%)  of  these  were 
from  the  experimental  group  and  8 
(27%)  from  the  control  group. 
Twenty-six  of  the  30  cases  are 
fully  documented  with  the 
remainder  in  various  stages  of 
follow-up.  Eight  of  the  thirty 
involved  medical  evacuation  from 
the  submarine  (four  each  from  the 
C  and  E  groups) . 

Table  I  shows  the  six  possible 
categories  of  diagnosis  with  the 
program.  Abbreviations  are 
arbitrary  and  are  those  used  in 
the  actual  computer  printout. 

Table  XI  summarizes  the  26 
documented  cases  in  approximate 
chronological  order.  The  "Initial 
Diagnosis"  and  "Computer  Diagnosis" 
are  made  at  the  time  of  presenta¬ 
tion  of  the  crewmember.  The  "Final 
Diagnosis"  is  made  one  week  after 
initial  presentation,  or  upon 
resolution  of  the  illness,  which¬ 
ever  is  later.  Hospital  data  is 
used  for  final  diagnosis  when  the 
crewmember  was  referred  for 
evaluation.  If  the  crewmember 
recovered  without  the  need  for 
hospital  or  physician  follow-up, 
the  final  diagnosis  is  made  from 
data  supplied  by  the  MDR. 

III.  PRELIMINARY  ANALYSIS  OF  DATA 

Table  III  summarizes  cases  as 
reported  with  the  totals  of  final 
diagnoses  shown. 

From  Table  II,  concordance 
between  the  MDR's  initial  diagnosis 
and  the  final  diagnosis  was  as 
follows:  Experimental  Group  (13/18), 
72%;  Control  Group  (7/8),  88%. 
Concordance  between  the  computer 
diagnosis  and  the  final  diagnosis 


was:  Experimental  Group  (13/18), 

72%;  Control  Group  (8/8),  100%. 
Concordance  between  the  MDR's 
initial  diagnosis  and  the  computer 
diagnosis  was:  Experimental 
Group  (17/18),  94%;  Control 
Group  (7/8),  88%. 

Table  IV  lists  the  18 
cases  from  Table  II  where 
medical  evacuation  was  not 
required.  From  Table  IV, 
concordance  between  the  MDR's 
initial  diagnosis  and  the  final 
diagnosis  was  as  follows: 

Experimental  Group  (12/14), 

86%;  Control  Group  (4/4),  100%. 
Concordance  between  the  computer 
diagnosis  and  the  final  diagnosis 
was:  Experimental  Group  (11/14), 

79%;  Control  Group  (4/4),  100%. 
Concordance  between  the  MDR's 
initial  diagnosis  and  the 
computer  diagnosis  was:  Experimental 
Group  (13/14),  93%;  Control 
Group  (4/4),  100%. 

Table  V  lists  the  8  cases 
from  Table  II  where  medical 
evacuation  was  required.  From 
Table  V,  concordance  between  the 
MDR's  initial  diagnosis  and  the 
final  diagnosis  was  as  follows: 
Experimental  Group  (1/4),  25%; 

Control  Group  (2/4),  50%. 

Concordance  between  the  computer 
diagnosis  and  the  final  diagnosis 
was:  Experimental  Group  (1/4), 

25%;  Control  Group  (3/4),  75%. 
Concordance  between  the  MDR's 
initial  diagnosis  and  the  computer 
diagnosis  was:  Experimental 
Group  (4/4),  100%;  Control 
Group  (3/4),  75%. 

These  tabulations  are  made 
for  the  reader's  convenience. 

The  limited  number  of  cases  makes 
any  conclusions  (positive  or 
negative)  about  the  impact  of  the 
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Table  I 


Categories  of  Diagnosis 


Diagnostic  Category 


Abbreviation 


Appendicitis  APPY 

Nonspecific  Abdominal  Pain  NONSAP* 

Renal  Colic  RCOLIC 

Perforated  Duodenal  Ulcer  PERFDU 

Cholecystitis  CROLE 

Small  Bowel  Obstruction  SMBOBS 


*Defined  as  abdominal  pain  that  is  non-surgical ,  not  life-threatening,  and 
not  requiring  medical  evacuation. 
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Table  II 


Case  Data  in  Approximate  Order  of  Submission 


Init ial 

Computer* 

Final** 

MEDEVAC 

Crewmember 

E/C 

Diagnosis 

Diagnosis 

Diagnosis 

Occurred 

1 

E 

RCOLIC 

RCOLIC 

OTHER  (UTI) 

NO 

2 

E 

RCOLIC 

RCOLIC 

RCOLIC 

NO 

3 

E 

NON SAP 

NON SAP  ■ 

NON SAP 

NO 

4 

E 

NON SAP 

NON SAP 

NON SAP 

NO 

5 

E 

NONSAP 

NONSAP 

NONSAP 

NO 

6 

E 

NON SAP 

NON SAP 

NONSAP 

NO 

7 

E 

APPY 

APPY 

NON SAP 

YES 

8 

C 

NON SAP 

{NONSAP  71%] 
{APPY  29%  ) 

NONSAP 

NO 

9 

C 

RCOLIC 

RCOLIC 

OTHER  (pyelo¬ 

YES 

nephritis) 

10 

E 

APPY 

APPY 

APPY 

YES 

11 

E 

NON SAP 

NONSAP 

OTHER  (bleeding 

YES 

ulcer) 

12 

E 

NON SAP 

NON SAP 

NON SAP 

NO 

13 

C 

NON SAP 

NONSAP 

NON SAP 

YES 

14 

c 

APPY 

NON SAP 

NONSAP 

YES 

15 

c 

APPY 

APPY 

APPY 

YES 

16 

E 

NONSAP 

NONSAP  (93%) 

NON SAP 

NO 

17 

E 

NONSAP 

NON SAP 

NON SAP 

NO 

18 

E 

NONSAP 

NONSAP 

NONSAP 

NO 

19 

E 

APPY 

APPY 

APPY 

NO 

20 

E 

NON SAP 

NON SAP 

OTHER 

NO 

(ulcerative 

colitis) 

21 

C 

NONSAP 

NONSAP 

NONSAP 

NO 

22 

E 

NON  SAP 

{APPY  86%  } 

{NONSAP  13%) 

NONSAP 

NO 

23 

C 

NONSAP 

NONSAP 

NON SAP 

NO 

24 

C 

RCOLIC 

RCOLIC 

RCOLIC 

NO 

25 

E 

APPY 

APPY 

APPY 

NO 

26 

E 

CHOLE 

CHOLE 

NON SAP 

YES 

*  The  computer  program  calculated  a  probability  >  95%  for  the  category 

specified  unless  noted  otherwise.  The  computer  diagnosis  is  not  available 
to  the  MDR  aboard  control  group  units;  it  is  calculated  by  NSMRL  from  the 
submitted  datasheet . 


**  If  a  specific  final  diagnosis  is  known,  but  it  is  not  among  the  five 
specific  categories,  it  has  been  labelled  "OTHER.”  This  applies  to 
cases  1,  9,  11,  20. 
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Table  III 


Final  Diagnosis  in  the  Experimental  and  Control  Groups 


Experimental  Group 


Control  Group 


APPY 

3 

1 

NONSAP 

11 

5 

RC0L1C 

1 

1 

PERFDU 

0 

0 

CHOLE 

0 

0 

SMBOBS 

0 

0 

OTHER 

3 

1 

MEDEVAC  OCCURRED 

4  (of  above) 

4  (of  above) 

5 


Table  IV 


! 


Data 

From  18  Cases  Not 

Requiring  Medical 

Evacuation 

CREWMEMBER  E/C 

Initial 

Diagnosis 

Computer 

Diagnosis 

Final 

Diagnosis 

1 

E 

RCOLIC 

RCOLIC 

OTHER  (UTI) 

2 

E 

RCOLIC 

RCOLIC 

RCOLIC 

3 

E 

NONSAP 

NON SAP 

NON SAP 

4 

E 

NONSAP 

NONSAP 

NON SAP 

5 

E 

NON SAP 

NONSAP 

NONSAP 

6 

E 

NONSAP 

NON SAP 

NON SAP 

12 

E 

NONSAP 

NONSAP 

NONSAP 

16 

E 

NONSAP 

NONSAP  (93%) 

NONSAP 

17 

E 

NONSAP 

NONSAP 

NON SAP 

18 

E 

NONSAP 

NON SAP 

NON SAP 

19 

E 

APPY 

APPY 

APPY 

20 

E 

NONSAP 

NONSAP 

OTHER  (Ulcerative  Colitis) 

22 

E 

NON SAP 

(APPY  86%  } 

{NONSAP  13%} 

NON SAP 

25 

E 

APPY 

APPY 

APPY 

8 

C 

NONSAP 

(NONSAP  71%) 
(APPY  29%  } 

NONSAP 

21 

C 

NON SAP 

NON SAP 

NONSAP 

23 

C 

NONSAP 

NONSAP 

NONSAP 

24 

C 

RCOLIC 

RCOLIC 

RCOLIC 

6 


Table  V 


Data 

from  8  Cases 

Requiring  Medical 

Evacuation 

CREWMEMBER 

E/C 

Initial 

Diagnosis 

Computer 

Diagnosis 

Final 

Diagnosis 

7 

E 

APPY 

APPY 

NON SAP 

10 

E 

APPY 

APPY 

APPY 

11 

E 

NONSAP 

NON  SAP 

OTHER  (Bleeding  Ulcer) 

26 

E 

CHOLE 

CHOLE 

NONSAP 

9 

C 

RCOLIC 

RCOLIC 

OTHER  (Pyelonephritis) 

13 

C 

NONSAP 

NON SAP 

NONSAP 

14 

C 

APPY 

NONSAP 

NON SAP 

15 

C 

APPY 

APPY 

APPY 

computer  program  quite  speculative. 
Several  times  this  number  of  cases 
would  be  required  to  make  such  an 
assessment . 

In  case  #11,  the  computer 
program’s  NONSAP  diagnosis  was 
incorrect.  The  MDR  agreed  with 
the  program’s  assessment  of  a 
benign  abdomen,  but  he  correctly 
evaluated  hematemesis  as  serious 
and  recommended  medical  evacuation. 
The  crewmember  recovered  ashore 
without  surgery  although  the 
administration  of  several  units 
of  blood  was  required.  In  the 
remainder  of  the  cases,  the 
computer  diagnosis  was  correct  or 
reasonable.  The  crewmember  in 
case  #7  had  an  appendectomy  ashore 
with  findings  of  a  normal  appendix. 
The  program  weighs  urinary  tract 
problems  (cases  #1  and  #9) 
towards  RCOLIC.  The  computation 
of  RCOLIC  by  this  program  is  an 


area  of  continuing  evaluation. 

IV.  PROBLEM  AREAS 

After  six  months  of  sea  trials, 
only  2  cases  of  abdominal  pain  had 
been  reported  to  NSMRL.  Periodic 
interviews  of  Squadron  Corpsmen 
disclosed  that  MDR's  in  the  Control 
Group  were  unhappy  with  their  role 
of  "data  collection"  without  the 
benefit  of  computer  output.  MDR's 
in  the  Experimental  Group,  while 
not  having  the  same  concern,  were 
unhappy  with  the  extra  paperwork 
requirements.  Some  MDR's  hadn't 
received  training  and  tapes  for 
several  months  after  the  study 
was  initiated.  There  was  a  general 
misunderstanding  about  just  which 
presentations  of  abdominal  pain 
were  to  have  a  datasheet  completed 
(i.e.,  all  presentations  or  just 
the  "serious"  cases?). 
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Beginning  in  February  1983, 
better  contact  with  Squadron 
Medical  personnel  was  established. 
A  letter  was  sent  to  all 
Squadron/Group  Commanders  and 
to  individual  submarine 
commanders  requesting  completion 
of  a  new  medical  evacuation 
form  (when  indicated)  and  noting, 
the  highlights  of  the  computer- 
assisted  diagnosis  program. 

It  was  requested  that  MDRs 
complete  a  datasheet  and  forward 
it,  along  with  supporting  SF-600 
entries  and  follow-up  (when 
possible)  to  NSMRL  for  all  cases 
of  acute  abdominal  pain,  severity 
unspecified.  Validation  of  the 
category  NONSAP  is  as  necessary 
as  validation  of  other  categories. 
The  training  program  was  modified 
to  allow  its  inclusion  in  the 
Control  Group  tapes.  Previously, 
the  training  program  could  have 
been  (hypothetically)  misused  to 
allow  a  diagnosis  of  a  real 
patient's  illness. 

In  May  1983,  onboard  visits  to 
New  London-area  submarines  were 
initiated.  The  program  objectives 
were  reinforced  with  the  actual 
user  with  distribution  of  updated 
tapes  occurring  at  these  visits. 

In  June  and  July  1983,  similar 
visits  to  all  submarine  ports  in 
CONUS  plus  Pearl  Harbor  were 
accomplished.  Approximately  55 
MDRs  were  interviewed,  onboard 
the  submarine  when  possible. 
Medical  log  books  or  sick  call 
log  books  were  reviewed  with  the 
MDR.  Instances  where  a  datasheet 
may  have  been  completed  were 
noted,  with  encouragement  given 
to  complete  one  for  similar  cases 
in  the  future.  Most  of  those 
interviewed  expressed  a  better 
understanding  of,  and  greater 
interest  in,  the  project 


afterwards. 

While  there  has  been  an  increase 
in  data  submitted  over  the  past 
9  months,  the  review  of  medical 
logs  or  sick  call  logs  showed 
that  about  10-20%  of  presenting 
cases  were  sent  to  NSMRL. 

Collection  of  data  at  this  rate 
would  yield  only  10-20%  of  the 
data  required  by  the  fifth  year 
of  the  study.  If  all  cases  . 
presenting  were  completed,  the 
amount  of  data  would  be  more  than 
adequate.  In  one  instance,  NSMRL 
learned  of  a  medical  evacuation 
for  appendicitis  from  the 
submarine's  Squadron  Corpsman.  No 
data  were  submitted.  When  NSMRL 
researchers  visited  that  submarine, 
there  was  a  new  MDR  aboard.  The 
crewmember  was  present  and  now  well 
after  his  appendectomy.  The 
datasheet,  which  is  essential,  was 
never  found,  having  been  apparently 
discarded . 

On  the  other  hand ,  there  have 
been  several  reports  of  MDRs 
utilizing  the  datasheet  to  assure 
a  complete  history  and  physical 
exam  of  the  abdomen  (both  in  the 
experimental  and  control  groups) . 
One  MDR  in  a  radio  message  to  a 
shore-based  facility  used 
terminology  directly  from  the 
datasheet  to  describe  his  ill 
crewmember's  condition.  He  was 
praised  for  having  sent  a  clear, 
concise,  complete  message  that 
left  out  no  essential  data.  A 
medical  evacuation  was  then 
recommended  by  the  Squadron  Medical 
Officer.  Several  MDRs  in  the 
experimental  group  have  expressed 
reassurance  in  having  the  computer 
program  reaffirm  their  initial 
diagnostic  impression.  The  value 
of  this  latter  point  in  both 
complicated  and  less  difficult 
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cases  should  not  be  underestimated. 


V.  PLANS 

A  vigorous  effort  to 
personally  interact  with  the 
user  of  the  system  will 
continue.  There  is  no 
impediment  to  this  in  the 
local  (New  London,  CT)  area. 
Distant  locations  require 
scarce  travel  funds  to  visit. 

Such  a  visit  is  planned  for 
FY84,  funds  permitting,  to 
interact  with  units  not  yet 
seen.  A  periodic  note  with 
highlights  of  the  system's 
usage  will  be  sent  to  all 
submarines.  The  crossover 
between  the  experimental  and 
control  groups  (one  for  one 
switch  of  roles  between  groups) , 
scheduled  for  December  1984,  may 
be  advanced  six  months.  The 
introduction  of  a  related  chest 
pain  system  will  be  pointed  out. 

VI.  OUTLOOK 

After  15  months  of  sea  trials, 
only  10-20%  of  the  number  of 
cases  expected  by  this  time  were 
received.  The  fact  that  more 
cases  were  available  means  that 
MDR  acceptance  of  the  entire 
system,  including  reporting 
requirements,  is  sub-optimal. 

It  is  obvious  that  the  control 
group  is  unhappy  with  their 
role,  but  there  is  no 
scientifically  acceptable  alter¬ 
native  to  a  control  group. 
Encouragement  and  personal 
contact  are  the  only  inducements 
to  compliance  that  can  be  offered 
outside  of  the  recent  addition 
of  the  training  program.  The 
requirement  from  the  Committee 
for  the  Protection  of  Human 
Subjects  to  designate  the  MDR 


as  the  "subject"  and  request 
completely  voluntary  cooperation, 
with  the  right  to  omit  any  aspect 
of  the  project,  hampers  our 
efforts.  It  is  recognized, 
however,  that  this  is  the  nature 
of  medical  research. 

The  anecdotal  experience  to 
date,  based  on  26  completely 
documented  cases,  suggests  that 
the  datasheet  serves  its  purpose 
well  and  that  the  diagnostic 
program  is  probably  valuable  as 
an  extended  decision  aid  to  the 
MDR. 
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